Image sensors include pixels. A pixel accumulates charge when illuminated by light. In conventional image acquisition methods, the pixel accumulates a charge in an analog circuit for a continuous period of time called an exposure time. The accumulated charge is transferred to an analog-to-digital (A/D) converter, which outputs a digital value for that pixel.
The image sensor outputs a two-dimensional array of digital values. Each digital value is obtained by the pixels in the image sensor. One type of image sensor is a charge-coupled device (CCD). In CCD image sensors, exposure of all pixels starts simultaneously and ends simultaneously.
In CMOS image sensors, exposure of all pixels in a same row starts simultaneously and ends simultaneously. In other image sensors, exposure of all pixels starts at the same time, but ends at different times.
What is common to conventional image sensors is that the digital value of a pixel is obtained as a result of A/D conversion of a charge that was accumulated during a continuous time interval of exposure.
FIG. 1 illustrates a conventional method of exposing a pixel to acquire an image. FIG. 1 illustrates an exposure time TExposure of 10 ms. The exposure time TExposure is continuous and, thus, is not interrupted. A value of a pixel is an accumulated charge during the continuous time interval of 10 ms. The accumulated charge is shown in a shaded region in FIG. 1.
Artifacts often result from conventional image acquisition methods, such as multiple exposure (ME) imaging and capturing images with a CMOS sensor under artificial illumination.
Multiple exposure (ME) allows production of an image at a higher dynamic range than what a sensor produces from a single capture. ME takes multiple captures of a same scene using different exposure times and then combines resulting images together into a wide dynamic range image. The process of capturing these images may be referred to as an “ME session”. Any motion in the captured scene during an ME session results in a (spatial) inconsistency between captured images, since, in a conventional model of image acquisition, each image is captured during the continuous time interval and images are taken in a sequence. This introduces significant time delay between exposure starting times for different images captured during the ME session. When separate images are combined into a wide dynamic range image, this inconsistency results in so-called “motion artifacts”.
A common method for image acquisition using a CMOS sensor is an electronic rolling shutter approach. Using this approach, exposure for each line of the CMOS sensor starts with a time delay relative to the previous line, and the readout of each line is delayed by the same amount of time relative to the previous line. The intensity of artificial illumination sources powered by alternating current changes over time. If the alternating current (AC) frequency is 50 Hz, the fluorescent lamp produces a waveform with a period of 100 Hz.
FIG. 2A illustrates a waveform of a “daylight” fluorescent lamp as measured with a light sensor and an oscilloscope. The Y-axis is in units of 20 mV and the X-axis is in units of 0.5 ms. The waveform can be represented as the sum of DC and AC components with an AC component that satisfies two criteria: (1) the waveform is close to periodic and (2) an integral over one period of the wave (e.g., 10 ms) is close to zero. The waveform is approximated by
                              f          ⁡                      (            t            )                          =                              C            1                    +                                    C              2                        ⁢                          sin              ⁡                              (                                                                                                    2                        ⁢                        π                                            T                                        ⁢                    t                                    +                  ϕ                                )                                                                        (        1        )            where C1 equals approximately 400 mV and C2 approximately equals 200 mV.
When capturing a scene illuminated by an artificial illumination source (with a non-zero AC component) with exposure not equal to a multiple of a flickering period, the integral of the AC component over a continuous exposure interval [t0,t1] less than the period of the AC component depends on the value of an AC component phase,
                                                        2              ⁢              π                        T                    ⁢          t                +        ϕ                            (        2        )            at time t0.
FIG. 2B illustrates a conventional approach to ME. As shown, each of a first image (Image 1) and a second image (Image 2) are collected during continuous intervals. The first and second images are then mixed to develop a wide dynamic range image.
In the conventional acquisition model, the exposure time interval is continuous, so when the exposure of each line of the image starts at different times, the AC component contributes different values to each line. Therefore, capturing a scene illuminated by an artificial illumination source (with a non-zero AC component) with exposure not equal to a multiple of a flickering period results in a flicker artifact visible on captured images in the form of alternating horizontal bars of a higher and lower brightness.
The flicker artifact is usually overcome in CMOS imagers by forcing an exposure time to be a multiple of the flicker period. In this case, the sinusoidal component does not affect the final image, since the integral of an AC component over an integer number of periods is close to zero. However, in many situations, exposure time that is less than a flicker period or not equal to the flicker period has to be chosen, for example to avoid saturation in the image.